Answer the issue, merge request, notification, push mirror and project rows of issue 1240 from GitLab's source - #1570

Every route that takes the project list filters declares
repository_checksum_failed and wiki_checksum_failed among
optional_filter_params_ee, and nothing reads them: the finder params the
routes build carry neither, and no finder, model or service names them.
The answer to the project rows of issue 1240 (#1570) stopped offering
them on the project lists for that reason, while ListProjectsOptions
still models both, so on main R-PARAM reports each as not_offered on
GET /projects, the forks, a user's projects and a user's starred
projects: eight findings no declaration answers, where this layer
leaves none.

Each is declared declared-not-applied, with the reason the structs
scope already records for leaving them out, and
TestRun_Repository_DeclaresTheChecksumFiltersNoProjectListReads holds
the eight to the declaration; its eight subtests fail without it. The
scope reports not_offered 0 again and 1491 declared (1510 params), and
request-paths.md says so.

Signed-off-by: José M. Requena Plens <mail@jmrp.io>

Every route that takes the project list filters declares
repository_checksum_failed and wiki_checksum_failed among
optional_filter_params_ee, and nothing reads them: the finder params the
routes build carry neither, and no finder, model or service names them.
The answer to the project rows of issue 1240 (#1570) stopped offering
them on the project lists for that reason, while ListProjectsOptions
still models both, so on main R-PARAM reports each as not_offered on
GET /projects, the forks, a user's projects and a user's starred
projects: eight findings no declaration answers, where this layer
leaves none.

Each is declared declared-not-applied, with the reason the structs
scope already records for leaving them out, and
TestRun_Repository_DeclaresTheChecksumFiltersNoProjectListReads holds
the eight to the declaration; its eight subtests fail without it. The
scope reports not_offered 0 again and 1491 declared (1510 params), and
request-paths.md says so.

Signed-off-by: José M. Requena Plens <mail@jmrp.io>

… here sends (R-PARAM) (#1639)

Refs #1595
Refs #1596
Refs #1598
Refs #1597
Refs #1599

This is the fifth layer of the auditor stack. A param GitLab declares on
a route we already call, and that client-go's option struct does not
model, was seen by no dimension: client-go cannot send it, so no input
offers it, so no recorded request carries it. The params of row 133 of
the upstream register came from reading the diff between two records by
hand.

## What the check detects

The first commit moves the option struct reader out of R-PATH into
`cmd/audit_1to1/internal/sdkoptions`, so the always-sent check and the
new scope read one answer. A field is read with the encoder its request
uses: `json` for a `NewRequest` PATCH, POST or PUT, `url` for every
other verb and for every field of an `UploadRequest`, whose constructor
`cmd/internal/sdkroutes` now records per method. The walk goes into any
client-go struct a field carries (`[]*HookCustomHeader` is
`custom_headers[key]`), and a type defined on an option struct
(`CreateProjectForUserOptions`) is read as that struct. The always-sent
check used to judge DELETE by the `json` tag and left GET unjudged; it
now walks 614 endpoints instead of 289 and still reports the same 11
rows.

The second commit adds `-scope=params`
(`cmd/audit_1to1/internal/params`). Per route, over the union of the
actions reaching it, a param is covered when a recorded request carries
it or a request sends it on every call. The inventory records the
top-level keys of a body, so a recorded key covers a param nested under
it only where an option struct of the route models that param. Otherwise
it is `unmodeled` (nothing here can send it), `not_offered` (the struct
the action already hands client-go models it and no input offers it),
`offered_unrecorded` (an input offers it and no recorded request carries
it) or `unjudged` (the options cannot be read). A raw request is read
through the static type of its options, which the sdk scope's walk now
exports (`sdk.RawRequests`), and the redirect GitLab answers another
route with carries no options of its own. An input offers a param by
name, through the requiredness scope's aliases, through its nested
schema, or flattened with `shared.NormalizeSDKTag` (`not[author_id]` as
`not_author_id`). Path params and the attributes Workhorse fills are not
judged.

The findings are a report. The gate fails only on a declaration that
answers nothing, names no category or reason, or names a register row
that does not exist; an `upstream` declaration answers an `unmodeled`
param alone, so it goes stale the day client-go models it. The scope
reads the committed request record, so CI runs it deferred with R-GRANT
(`make audit-1to1-params`) and it is step 30 of `make analyze`.

## Witness test, red then green

`TestRun_Repository_ReportsTheParamsClientGoV3300CannotSend` pins what
the scope sees on v3.30.0: `source_project_id` and the twelve other new
filters on the four merge request lists, the deployment one included;
the system hooks' `url_variables` and `custom_headers`; `url_variables`
on the project and group hooks; the three `duo_*` flags of `PUT
/projects/:id`; `size` on a registry repository; `not[author_id]` on the
three `issues_statistics` routes and the group epics;
`include_html_description` on a release read, which `GetRelease` takes
no options to send; `approval_rules[id]` of a protected environment,
nested under the `approval_rules` the inventory records and modeled by
no field of `EnvironmentApprovalRuleOptions`; the declared `external`;
the four `duo_*` flags GitLab drops on both create routes; and the four
label collection routes as `branch-not-taken`. It also holds that a
project hook add's custom header keys are covered only through
`AddProjectHookOptions`, the struct its raw request sends, and that the
edit's stand under the `custom_headers` container
`EditProjectHookOptions` models. The bump stack changes it, which is the
proof the scope ran again.

A copy that reads the report as plain JSON fails on the parent
(47f2535a0), where no scope reports the param (`paths` names
`source_project_id` only as a response key, under `shapes`):

```text
$ go test -count=1 -run TestZZWitness -v ./cmd/audit_1to1/
    zz_witness_test.go:22: the params scope did not run: invalid scope "params" (valid: structs, actions, metadata, enums, sdk, paths, grants, required, all); nothing reports a param client-go cannot send
--- FAIL: TestZZWitness_SourceProjectIDOnTheFourMergeRequestLists (0.00s)
```

and passes here:

```text
    zz_witness_test.go:48: GET /merge_requests source_project_id unmodeled
    zz_witness_test.go:48: GET /projects/:id/merge_requests source_project_id unmodeled
    zz_witness_test.go:48: GET /groups/:id/merge_requests source_project_id unmodeled
    zz_witness_test.go:48: GET /projects/:id/deployments/:deployment_id/merge_requests source_project_id unmodeled
--- PASS: TestZZWitness_SourceProjectIDOnTheFourMergeRequestLists (3.68s)
```

The stand-in tests hold both client-go states: a list option without
`merged_after` gives the finding and its `upstream` declaration answers
it; with the field it turns `not_offered`, the declaration goes stale
and the gate fails. They also hold that a member nested under a recorded
key that the stand-in's struct does not model is `unmodeled`, and that a
redirect another action follows leaves its route's params judged.

## What it finds on main, and how each is answered

On the 19.5.0 record it judges 5791 params of 1101 reached routes and
covers 2792. The summary counts findings, and a container that stands
for its members is one finding: 785 `unmodeled` on 168 routes (905
params), 79 `not_offered` on 13 (85), 523 `offered_unrecorded` on 98
(537), 3 `unjudged`, and 1445 declared (1458).

Declared: the settings update and every integration take a map
(`map-input`, 1371), rows 59 and 133 of the register (`upstream`, 42),
four `duo_*` flags GitLab drops on create (`declared-not-applied`, 8,
from `filter_duo_vulnerability_attributes!`), the GitHub payload of the
mirror pull (`webhook-payload`, 8), and every param of the four label
collection routes, which client-go takes only for a nil label argument
and our handlers never pass one (`branch-not-taken`, 16).

The 79 `not_offered` are what the next layer answers (#1596): 61
`override_params[...]` of the file import, three list flags of the
deployment merge requests, the avatar on eight create and edit actions,
and `repository_checksum_failed` and `wiki_checksum_failed` on the four
project lists, which main stopped offering in #1570 because GitLab reads
neither. The `unmodeled` backlog goes to client-go (#1598), and gating
the undeclared findings waits on it (#1599).

One finding is a defect of ours, and I left it undeclared so it stays
visible: `admin.terraform_state_lock` sends none of the seven lock
fields GitLab requires, because client-go's `TerraformStates.Lock` takes
no options, so every call is refused with `ID is missing` (#1597). No
category for a protocol body is added: the one candidate is that defect,
left visible.

The figures `request-paths.md` and `cmd-utilities.md` give in this layer
are the ones I measured before the stack was rebased onto main (613
endpoints for the always-sent check, and for R-PARAM 2794 covered, 71
`not_offered` on 9 routes and 534 `offered_unrecorded` on 99); main's
newer request record and #1570 have moved them since. The top of the
stack writes there what the merged tree measures.

## Checks

`go test ./... -count=1` with the snapshot parity deferred, and every
`cmd/audit_1to1` suite on the real tree without it. 100% statements and
every condition both ways (gobco) on `sdkoptions`, `params`,
`sdkroutes`, `paths`, `required`, `sdk` and the command. gremlins: Lived
0 and Not covered 0 on each, except `paths` (536 killed), whose 490 not
covered are package-level initializers in seven files of `paths` this
layer does not touch (`endpoint_declarations.go`, `endpoints.go`,
`orbit_declarations.go`, `pagination_declarations.go`, `published.go`,
`sent_declarations.go` and `shape_declarations.go`), which the coverage
profile does not mark. The touched `make analyze` steps pass, and
`check-action-grants-derivation` is unchanged.

---------

Signed-off-by: José M. Requena Plens <mail@jmrp.io>

…ady models, and declare the ones GitLab drops (R-PARAM) (#1640)

Closes #1596
Refs #1598
Refs #1595

This layer acts on what R-PARAM found that this repository can fix
today. On its parent the scope lists 79 `not_offered` findings on 13
routes: params the option struct an action already hands client-go
models, and that no input offers. I answered every one, offering what
GitLab applies and declaring what GitLab drops or cannot apply. A
witness test pins 71 of them to that answer, and a second one the eight
checksum filters of the project lists, which main stopped offering in
#1570.

## What the check detects

`TestRun_Repository_OffersOrDeclaresTheParamsNoInputOfferedOnV3300`
reads the R-PARAM report of the real tree and pins 71 findings of
v3.30.0: the avatars, the 61 `override_params` keys and the three list
flags. Each param that is offered now has to be offered and answered by
no declaration, and each one that is declared has to be
`declared-not-applied` on a param client-go models and no input offers,
`lfs_enabled` among them, which the import input offered before this
stack. It is a witness and not a gate on the class: R-PARAM stays a
report with its gate on stale declarations, as decided, so a param
client-go starts modeling in the bump is reported as `not_offered`
beside the rest and the bump updates this list the way it updates every
witness.

The layer also stops a `declared-not-applied` or `webhook-payload`
declaration from answering a param an input offers. Before, it answered
a finding of any class, so it could hide the very defect this layer
fixes: had `lfs_enabled` come back into the import input, the
declaration would have answered the `offered_unrecorded` finding in
silence. Now the declaration goes stale and the gate fails, which I
checked by putting the field back: `POST /projects/import
override_params[lfs_enabled] (declared-not-applied): answers no
finding`.

## Witness test, red then green

On the parent (2c79544e2, with this test file and nothing else of the
layer) 72 subtests of the witness fail, the 46 params that are offered
here and the 26 that are declared, and so do the eight of
`TestRun_Repository_DeclaresTheChecksumFiltersNoProjectListReads`, one
per list and filter:

```text
$ go test -count=1 -run 'TestRun_Repository_OffersOrDeclaresTheParamsNoInputOfferedOnV3300|TestRun_Repository_DeclaresTheChecksumFiltersNoProjectListReads' ./cmd/audit_1to1/internal/params/
    analyze_test.go:622: POST /groups avatar = {Route:POST /groups Param:avatar Class:not_offered SDKModels:true Offered:false ...}, want it offered and answered by no declaration
    analyze_test.go:629: POST /projects/import override_params[topics] = {... Class:not_offered SDKModels:true Offered:false ...}, want the modeled param declared declared-not-applied and offered by no input
    analyze_test.go:629: POST /projects/import override_params[lfs_enabled] = {... Class:offered_unrecorded SDKModels:true Offered:true ...}, want the modeled param declared declared-not-applied and offered by no input
    analyze_test.go:656: GET /projects repository_checksum_failed = {Route:GET /projects Param:repository_checksum_failed Class:not_offered SDKModels:true Offered:false ...}, want the modeled param declared declared-not-applied and offered by no input
FAIL	github.com/jmrplens/gitlab-mcp-server/v3/cmd/audit_1to1/internal/params
```

and both pass here:

```text
--- PASS: TestRun_Repository_OffersOrDeclaresTheParamsNoInputOfferedOnV3300
--- PASS: TestRun_Repository_DeclaresTheChecksumFiltersNoProjectListReads
ok  	github.com/jmrplens/gitlab-mcp-server/v3/cmd/audit_1to1/internal/params
```

The class restriction has its own unit cases, red before it (`not
applied on an offered param` and `webhook payload on an offered param`,
`answers = true, want false`) and green after.

## What it found on main, and how each is answered

The avatar on seven create and edit routes: `group.create`,
`project.create`, `project.create_for_user`, `user.create`,
`user.modify`, `admin.topic_create` and `admin.topic_update` now take an
`avatar` object (the file name GitLab reads the image type from, and a
local file or base64 content), refused before any request when it cannot
be read. With it client-go sends the call as a multipart upload.
`interactive.project_create` reaches the same route, which
`project.create` now covers.

That upload writes only the first value of each field (`UploadRequest`,
`fields.Get`). So beside an avatar the project create sends its topics
and tags, and the group create its download limit allowlist, as one
comma-separated value, which GitLab splits back
(`CommaSeparatedToArray`). The group alert list is an `Array[Integer]`
GitLab does not split, so a create that gives one beside an avatar is
refused and pointed at `group.update`. The test that holds the allowlist
was red on the commit before the fix: the mock read `["alice"]` for
`alice` and `bob`. The defect is now row 151 of the upstream register,
with the three sites. The upstream wait table cannot hold it, because
its predicates read types and routes and this is the body of a function,
so `TestSDK_CreateProjectWithAvatar_WritesTheFirstTopicAlone` pins what
client-go sends and fails the day it sends more.

The 61 `override_params` keys of `project.import_from_file`: I read the
import in GitLab's source. The overrides reach the project only through
the tree restorer, which keeps the attributes
`lib/gitlab/import_export/project/import_export.yml` permits for a
project (`AttributesPermitter`, through
`Group::RelationTreeRestorer#filter_attributes`), and then hands them to
`Project#assign_attributes`. Of the 86 keys GitLab declares there it
permits 52, and two of those it cannot assign: `topics` is the project's
association with `Projects::Topic`, which a list of names does not fill,
and the project has no `jobs_enabled` writer at all, since
`lib/api/projects.rb` renames it to `builds_enabled` on the create and
the edit and never on the import. Either one raises inside the restore,
which fails the import. That is a trace of GitLab's code, which I did
not run against an instance; no e2e scenario imports with overrides. The
input now offers the 50 keys the import applies, 39 of the 61 among them
and 11 it offered before. The other 22 of the 61 are declared
`declared-not-applied`, and so are `lfs_enabled` and the 13 dropped keys
client-go does not model, 36 in all, so none of them waits on a
client-go change that would not make GitLab apply it. The input also
stopped offering five keys it offered before this stack: `name`, `path`,
`namespace_id` and `default_branch`, which Grape drops because the
import does not declare them under `override_params`, and `lfs_enabled`,
which the restorer excludes by name.

The three list flags of a deployment's merge requests (`view`,
`with_labels_details`, `with_merge_status_recheck`) are declared
`declared-not-applied`: `lib/api/deployments.rb` presents
`MergeRequestBasic` with no serializer options, so the route applies
none of them.

Two things came out of the work. `PUT /groups/:id`, `PUT /projects/:id`
and `PUT /topics/:id` remove the avatar when it is sent blank, which
client-go writes for an empty avatar, and no action did it:
`group.update`, `project.update` and `admin.topic_update` take
`remove_avatar`. And the request inventory now records the part names of
a multipart form, so an upload's options can cover a param the way a
JSON body's keys do.

## The report after

On the committed request record the scope reports `not_offered` 0 (was
79), `unmodeled` 773 (was 785) and 1491 declared (was 1445). Eight of
the declarations answer `repository_checksum_failed` and
`wiki_checksum_failed` on the four project lists, which main stopped
offering in #1570 because GitLab reads neither, and which R-PARAM would
otherwise report `not_offered`;
`TestRun_Repository_DeclaresTheChecksumFiltersNoProjectListReads` holds
them. Of the 13 dropped import keys client-go does not model, 12 left
`unmodeled` for their declaration; the thirteenth,
`duo_code_review_decisions_enabled`, was already answered by row 133,
whose text now says the import would not apply it. The params offered
here show as `offered_unrecorded` (568 on 103 routes, was 523 on 98)
until the inventory is regenerated, which the stack does once at its
top. There the recorded requests carry every param this layer offers:
covered goes from 2792 to 2864, `offered_unrecorded` falls to 496 on 92
routes, and no avatar or `override_params` key the layer offers is left
among the findings. The R-PARAM figures `request-paths.md` gives in this
layer (2794 covered, 579 `offered_unrecorded` on 104) are the ones I
measured before the stack was rebased onto main, and the top writes
there what the merged tree measures.

The `unmodeled` backlog, grouped by client-go service, is #1598.

## Checks

`go test ./... -count=1` with the snapshot parity deferred, and every
`cmd/audit_1to1` suite on the real tree without it. 100% statements and
every condition both ways (gobco) on the nine touched packages. gremlins
on the files the layer changes, the rest of each package excluded: Lived
0 and Not covered 0 in `topics`, `users`, `groups`, `projects`,
`projectimportexport`, `params` and `testutil`. The first run left one
mutant alive in the multipart recorder (`||` to `&&` on the boundary
check), and its test now holds that a form without a boundary is refused
before its body is read; `packageOutside`, a tagless switch from #598
whose case expressions gremlins could not see, is a chain of ifs now,
which took `testutil` from 3 not covered to none. The `structs` and
`adminspecs` changes are table entries and a string. `golangci-lint` is
clean on them, and the `make analyze` steps the layer touches pass, as
do markdownlint and the doc link check on the four contributor pages it
edits. The new end-to-end scenarios compile and `make check-e2e-static`
passes; I did not run them against a GitLab here. `gen_testing_docs
--check` reports the test counts stale, as it already did below this
layer, and is regenerated at the top.

---------

Signed-off-by: José M. Requena Plens <mail@jmrp.io>